The present invention relates to a process for computerized grading of formula-based multi-step problems via a web-interface. At present, it is generally known to grade math, science, engineering and/or technology problems via the use of a global computer network through access to websites. Such services are currently being rendered by distance education providers such as Blackboard, Brownstone, WebCT, ecollege and ANGEL. Major textbook publishers have created websites for the same purpose. Applicant is aware of publishers John Wiley and Sons, Brooks/Cole, Addison Wesley, and Prentice Hall establishing websites for this purpose. Additionally, academic institutions have been offering computerized grading of problems, both alone and in partnership with distance education providers or computer algebra system (CAS) vendors. Applicant is aware that two major such vendors, Maple and Mathematica, have developed working relationships with distance education providers, academic institutions, and publishers.
Systems developed by Maple and Mathematica can manipulate mathematical formulas and can recognize any numerical or symbolic expression that is likely to be an answer to any problem in any course one may encounter at the pre-college, undergraduate, or graduate level. As a result, the most intense current activity in computer-grading of math-type problems involves grading answers that are symbolic rather than multiple choice type answers, fill-in-a-blank type answers or purely numeric answers. The significant features employed by known symbolic answer graders are the following:
(1) A variety of methods for free-form entry of symbolic-expression answers;
(2) Gradable problems have parameters that can be used to change numerical values in the problem statement. Assignment of parameters can be random;
(3) The use of computer algebra systems to grade problems by comparing a submitted answer to a pre-programmed correct answer, with checks to ensure that an answer is of the correct type, for example, algebraic category, and that students submit an answer rather than computer-algebra-system instructions to achieve the answer;
(4) Intelligent hints to permit algorithmic, interactive self-grading of an answer;
(5) Sub-questions employed as hints to the student if the student wishes to use them;
(6) The ability to resubmit incorrect answers repeatedly until a correct answer is achieved with provision for penalties for multiple tries to correctly answer a problem;
(7) Use of predefined answer types such as, for example, algebraic, matrix, constant, multiple choice, or string;
(8) Systems are known that permit both students and instructors to obtain detailed progress reports.
Distance education providers emphasize customization of problems to meet the desires and needs of an individual instructor.
The problems that typically appear in course textbooks for algebra, calculus, and other mathematical and technical subjects are formula-based multi-step problems for which answers to earlier steps are used to obtain subsequent intermediate and final answers. Moreover, a particular intermediate answer may be obtainable from prior answers in several different ways, each of which may involve an equally valid methodology. Many, if not most, problems assigned for homework or given on a quiz or exam in such courses are typically adapted from a textbook and hence are of this multi-step form.
As should be understood from the above explanation, current web-based systems grade only one answer at a time. Related answers are requested diagnostically, for example, as a hint or sub-problem, only after a single answer is recognized as incorrect. As such, in such systems, the student must submit all correct answers, one at a time, to obtain full credit.
By contrast, effective hand-grading of formula-based multi-step problems uses two shortcut-type strategies, referred to herein as the Prior-Answer Strategy and the Later-Answer Strategy:                (1) The Prior-Answer Strategy looks for correct intermediate or final answers, which when found give full credit for all relevant prior answers used to get it.If an incorrect intermediate answer is found, an experienced hand-grader will use the second strategy;        (2) The second strategy is the Later-Answer Strategy in which subsequent steps are examined and subjective credit is given for answers that are incorrect, but which appear to have been obtained correctly based upon incorrect but reasonable answers to earlier steps.        
When hand-graders apply these two strategies combined with the innate human ability to recognize and accept more than one correct formula for using prior answers to obtain later answers, the student obtains the maximum credit deserved for problem steps that were correctly performed. The prior art fails to teach or suggest any computerized system that incorporates these grading strategies used by effective hand-graders to fairly and fully grade problems. It is with this thought in mind, among others, that the present invention was developed.